Parallax Geometry of Pairs of Pointsfor 3 D Scene Analysis 1 Michal

We present a geometric relationship between the image motion of pairs of points over multiple frames. This relationship is based on the parallax displacements of points with respect to an arbitrary planar surface, and does not involve epipolar geometry. A constraint is derived over two frames for any pair of points, relating their projective structure (with respect to the plane) based only on their image coordinates and their parallax displacements. Similarly, a 3D-rigidity constraint between pairs of points over multiple frames is derived. This leads to the derivation of an image point which is dual to the epipole. We show applications of these parallax-based constraints to solving three important problems in 3D scene analysis: (i) the recovery of 3D scene structure, (ii) the detection of moving objects in the presence of camera induced motion, and (iii) the synthesis of new camera views based on a given set of views. Moreover, we show that this approach can handle diicult situations for 3D scene analysis, e.g., where there is only a small set of parallax vectors, and in the presence of independently moving objects.